In many industrial, commercial, and scientific applications, it is critical to precisely synchronize multiple devices or control the multiple devices to operate in a coordinated manner. For example, to create an immersive virtual-reality (VR) experience, multiple cameras may be used to cover different parts of a scene. Image frames captured by the cameras at the same time are synthesized to achieve a three-dimensional (3D) reconstruction of the scene. When the scene is dynamic, i.e., containing moving objects, a well-known challenge is precisely synchronizing the image capturing at frame level or even sub-frame level (i.e., a fraction of the duration of a frame).
A key task in immersive VR production is accurately reconstructing 3D object shape, motion, and surface texture from multi-view 2D image data. Similar to human eyes seeing objects at the right depth through binocular vision, multiple cameras may be used to shoot images of an object from different viewpoints. For an object in motion, unless the images with different viewpoints are captured at the same time, no proper stereoscopic display of the object can be generated.
FIG. 1 is a schematic diagram illustrating a multi-camera system for obtaining 3D image information of an object. Referring to FIG. 1, two cameras, i.e., cameras 1 and 2, may be separated apart by a predetermined distance and aligned to shoot a moving car. The differences between the images taken by the two cameras at the same time may be interpreted to determine the depth information of the car. However, if cameras 1 and 2 are unsynchronized, the image frame captured by camera 1 at time T1 cannot be combined with the image frame captured by camera 2 at time T2 to construct a virtual scene showing the right depth.
Moreover, a viewer of VR is free to look anywhere in the virtual world. Therefore, VR needs to provide a wide field of view (FOV), such as a 360-degree spherical view. Since a camera only has a limited FOV, images shot by multiple camera may be combined into a panorama. FIG. 2 is a schematic diagram illustrating a multi-camera system for obtaining a panorama. Referring to FIG. 2, cameras 1 and 2 have partially overlapping FOVs. Each FOV covers part of a scene including a moving car traveling past several motionless tires. If the two cameras are unsynchronized, when the car is traveling in the overlapping area of the FOVs, the image frame captured by camera 1 at time T1 and the image frame captured by camera 2 at time T2 show the car camera 1 at different pixels. Such pixel difference is usually called the “ghosting” artifact. That is, the panorama generated from these two image frames may show the moving car as double images, or “ghosts,” at the area where the two frames overlap. In contrast, because the still tires always stay at the same pixels, the tires have no ghosts.
The disclosed methods and systems address one or more of the problems listed above.